Uczenie Maszynowe - LAB2b - LIME¶
Biblioteka LIME: https://github.com/marcotcr/lime (Dokumentacja API: https://lime-ml.readthedocs.io/en/latest/)
Wprowadzenie - pakiety¶
NiezbÄdne pakiety i moduÅy na potrzeby wprowadzenia
import json
from functools import partial
import matplotlib.pyplot as plt
import numpy as np
import torch
from lime import lime_image
from PIL import Image
from skimage.segmentation import mark_boundaries
from torchvision import models, transforms
Wprowadzenie - funkcje pomocnicze¶
Funkcja do wczytywania wskazanego obrazka oraz konwersji do palety RGB.
def get_image(path):
with open(path, 'rb') as f:
with Image.open(f) as img:
return img.convert('RGB')
Funkcja do przeksztaÅcania obrazka (zwróconego przez funkcjÄ get_image) w tensor, akceptowalny na wejÅciu sieci neronowej.
def image_to_tensor(img):
transformer = transforms.Compose([
transforms.Resize((256, 256)),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
return transformer(img).unsqueeze(0)
Funkcja używana przez LIME, przyjmuje na wejÅciu zbiór obrazków, a zwraca prawdopodobieÅstwa klas. Należy jÄ
przekazaÄ do lime_image.LimeImageExplainer().explain_instance przy użyciu partial, jako partial(predict_batch, <model>), gdzie modelem w naszym wypadku bÄdÄ
sieci neuronowe. PrzykÅady użycia sÄ
zawarte w tym notebooku.
def predict_batch(model, images):
model.eval()
transformer = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
model.eval()
batch = torch.stack(tuple(transformer(i) for i in images), dim=0)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
batch = batch.to(device)
logits = model(batch)
probas = torch.nn.functional.softmax(logits, dim=1)
return probas.detach().cpu().numpy()
Funkcja, która przeksztaÅca obrazek w format akceptowany na wejÅciu przez LIME. PrzykÅady użycia sÄ zawarte w tym notebooku.
def lime_transformer(image):
transformer = transforms.Compose([
transforms.Resize((256, 256)),
transforms.CenterCrop(224)
])
return np.array(transformer(image))
LIME jest gÅównie wykorzystywane do wyjaÅniania predykcji tzw. czarnych skrzynek, czyli modeli nieinterpretowalnych. Idealnymi kandydatami sÄ GÅÄbokie Sieci Neuronowe, dlatego spróbujemy wyjaÅniÄ niektóre predykcje gotowych modeli.
Model Inception-v3 - przygotowanie danych¶
Plik ./data/imagenet_class_index.json zawiera przypisanie klas obrazków do indeksów. Jest to istotne, ponieważ zwracane wyniki (np. wartoÅci funkcji logit na wyjÅciu sieci neuronowych) wykorzystujÄ
to, zwracajÄ
c wyniki w zadanej kolejnoÅci.
with open("./data/imagenet_class_index.json") as f:
content = json.load(f)
index_to_label = {
int(index): data[1]
for index, data in content.items()
}
image_to_classify = get_image("./data/dogs.png")
plt.imshow(image_to_classify)
<matplotlib.image.AxesImage at 0x2428a17dcd0>
img_tensor = image_to_tensor(image_to_classify)
ZaÅadowanie pretrenowanego modelu¶
inception_v3 = models.inception_v3(pretrained=True)
c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=Inception_V3_Weights.IMAGENET1K_V1`. You can also use `weights=Inception_V3_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg) Downloading: "https://download.pytorch.org/models/inception_v3_google-0cc3c7bd.pth" to C:\Users\Monika/.cache\torch\hub\checkpoints\inception_v3_google-0cc3c7bd.pth 100%|āāāāāāāāāā| 104M/104M [00:02<00:00, 54.2MB/s]
Predykcja¶
inception_v3.eval()
logits = inception_v3(img_tensor)
ZwrĆ³Ä uwagÄ, że model zwraca wartoÅci funkcji logit, a nie prawdopodobieÅstwa klas, dlatego wyniki trzeba przetworzyÄ (np. przy użyciu funkcji softmax).
probas = torch.nn.functional.softmax(logits, dim=1)
SprawdÅŗmy N najbardziej prawdopodobnych klas
TOP_N_LABELS = 15
probas_top = probas.topk(TOP_N_LABELS)
top_probas = probas_top[0][0].detach().numpy()
top_labels = probas_top[1][0].detach().numpy()
for proba, label in zip(top_probas, top_labels):
print(f"Class: {index_to_label[label]:<30} | Probability: {proba:.6f}")
Class: Bernese_mountain_dog | Probability: 0.935930 Class: EntleBucher | Probability: 0.038448 Class: Appenzeller | Probability: 0.023756 Class: Greater_Swiss_Mountain_dog | Probability: 0.001818 Class: Gordon_setter | Probability: 0.000009 Class: Blenheim_spaniel | Probability: 0.000007 Class: English_springer | Probability: 0.000002 Class: tabby | Probability: 0.000002 Class: robin | Probability: 0.000001 Class: guinea_pig | Probability: 0.000001 Class: amphibian | Probability: 0.000001 Class: Japanese_spaniel | Probability: 0.000001 Class: African_grey | Probability: 0.000001 Class: Brittany_spaniel | Probability: 0.000001 Class: toucan | Probability: 0.000001
Teraz możemy te funkcje zebraÄ razem¶
def get_prediction_probabilities(image, model):
img_tensor = image_to_tensor(image)
model.eval()
logits = model(img_tensor)
probas = torch.nn.functional.softmax(logits, dim=1)
TOP_N_LABELS = 15
probas_top = probas.topk(TOP_N_LABELS)
top_probas = probas_top[0][0].detach().numpy()
top_labels = probas_top[1][0].detach().numpy()
for proba, label in zip(top_probas, top_labels):
print(f"Class: {index_to_label[label]:<30} | Probability: {proba:.6f}")
I sprawdziÄ jak ta predykcja wyglÄ da dla innego obrazka¶
exercise_image = get_image("./data/cat_mouse.jpeg")
plt.imshow(exercise_image)
<matplotlib.image.AxesImage at 0x2428a6ca510>
Zadanie: sprawdÅŗ jak bÄdzie wyglÄ daÅa predykcja dla powyższego obrazka¶
get_prediction_probabilities(exercise_image, inception_v3)
Class: Egyptian_cat | Probability: 0.967492 Class: tabby | Probability: 0.024167 Class: lynx | Probability: 0.005490 Class: tiger_cat | Probability: 0.002165 Class: Persian_cat | Probability: 0.000105 Class: Angora | Probability: 0.000074 Class: swab | Probability: 0.000071 Class: Madagascar_cat | Probability: 0.000064 Class: snow_leopard | Probability: 0.000040 Class: tile_roof | Probability: 0.000037 Class: indri | Probability: 0.000020 Class: leopard | Probability: 0.000016 Class: Siamese_cat | Probability: 0.000011 Class: ram | Probability: 0.000010 Class: crate | Probability: 0.000009
Model Inception-v3 - wyjaÅnienie¶
Chcemy wiedzieÄ dlaczego klasa Bernese_mountain_dog zostaÅa uznana przez sieÄ neuronowÄ
za najbardziej prawdopodobnÄ
(to znaczy - które piksele obrazka o tym zadecydowaÅy). W tym celu wÅaÅnie wykorzystamy LIME.
W jaki sposób dziaÅa LIME na obrazkach?
- Na wejÅciu wymagany jest oryginalny obrazek.
- WejÅciowy obrazek jest delikatnie przeksztaÅcany wiele razy, dziÄki czemu otrzymujemy wiele podobnych (ale nie takich samych!) obrazków.
- Dodatkowo na wejÅcie musimy podaÄ funkcjÄ, która każdemu takiemu przeksztaÅceniu nada prawdopodobieÅstwo przynależnoÅci do danej klasy. Jest to wymagane ponieważ LIME jest niezależny od żadnych narzÄdzi i modeli.
explainer = lime_image.LimeImageExplainer()
explanation = explainer.explain_instance(
image=lime_transformer(image_to_classify),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
MajÄ c te dane możemy teraz sprawdziÄ które kategorie sÄ najbardziej prawdopodobne
for index in explanation.top_labels:
print(index_to_label[index])
Bernese_mountain_dog EntleBucher Appenzeller Greater_Swiss_Mountain_dog Gordon_setter
Zobaczmy co wpÅynÄÅo na wybranie Bernese_mountain_dog jako najbardziej prawdopodobnej klasy.
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x2428a6deed0>
Zadanie: zmieÅ wartoÅÄĀ NUM_FEATURES i zaobserwuj jak zmienia siÄ mapowanie¶
NUM_FEATURES najlepiej zmieniaÄ w zakresie 1:50
NUM_FEATURES = 10
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=NUM_FEATURES,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x2428a769010>
NUM_FEATURES = 30
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=NUM_FEATURES,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x2428a7b8110>
NUM_FEATURES = 1
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=NUM_FEATURES,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x2428a6ca8a0>
The bigger NUM_FEATURES is the bigger area is marked. For too small value unimportant area is marked.
Zielone fragmenty oznaczajÄ "superpiksele", które pozytywnie wpÅywajÄ na predykowanÄ klasÄ. Czerwone fragmenty wpÅywajÄ negatywnie.
Zobaczmy jak to siÄ prezentuje dla drugiej najbardziej prawdopodobnej klasy, czyli EntleBucher, która jednak otrzymaÅa jedyne 3.8%.
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[1],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x2428fd0e360>
UstawiajÄ
c wartoÅci hide_rest oraz positive_only na True jesteÅmy w stanie zostawiÄ tylko te piksele, które potwierdzaÅy przynależnoÅÄ do danej klasy
Musimy jednak pamiÄtaÄ o przeskalowaniu rezultatu przy pomocy (boundaries).astype(np.uint8)
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=True,
negative_only=False,
num_features=10,
hide_rest=True)
boundaries = mark_boundaries(image, mask)
plt.imshow((boundaries).astype(np.uint8))
<matplotlib.image.AxesImage at 0x2428fcf3740>
Możemy również zostawiÄ tylko te piksele, które zaprzeczaÅy przynależnoÅci do danej klasy
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=True,
num_features=10,
hide_rest=True)
boundaries = mark_boundaries(image, mask)
cropped_image_ndarray = (boundaries).astype(np.uint8)
plt.imshow(cropped_image_ndarray)
<matplotlib.image.AxesImage at 0x24290101c10>
A nastÄpnie sprawdziÄĀ co model sÄ dzi o tak wyciÄtym obrazku
cropped_image_pil = Image.fromarray(cropped_image_ndarray)
get_prediction_probabilities(cropped_image_pil, inception_v3)
Class: lumbermill | Probability: 0.272693 Class: triceratops | Probability: 0.241274 Class: pedestal | Probability: 0.118357 Class: sundial | Probability: 0.081880 Class: fountain | Probability: 0.029933 Class: caldron | Probability: 0.029304 Class: African_grey | Probability: 0.018543 Class: vulture | Probability: 0.017247 Class: stone_wall | Probability: 0.013481 Class: cannon | Probability: 0.011382 Class: black_grouse | Probability: 0.009726 Class: frilled_lizard | Probability: 0.009235 Class: chain_saw | Probability: 0.007655 Class: grand_piano | Probability: 0.007258 Class: conch | Probability: 0.006042
I jak go teraz widzi model
cropped_image_explanation = explainer.explain_instance(
image=lime_transformer(cropped_image_pil),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
image, mask = cropped_image_explanation.get_image_and_mask(
label=cropped_image_explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x2428a6c9880>
Model Inception-v3 - porównanie z AlexNet¶
Przetestujmy dziaÅanie na innym modelu - AlexNet
alexnet = models.alexnet(pretrained=True)
c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg) Downloading: "https://download.pytorch.org/models/alexnet-owt-7be5be79.pth" to C:\Users\Monika/.cache\torch\hub\checkpoints\alexnet-owt-7be5be79.pth 100%|āāāāāāāāāā| 233M/233M [00:05<00:00, 47.6MB/s]
explanation_alexnet = explainer.explain_instance(
image=lime_transformer(image_to_classify),
classifier_fn=partial(predict_batch, alexnet),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
for index_alex, index_inception in zip(explanation_alexnet.top_labels, explanation.top_labels):
print(f"{index_to_label[index_alex]:30} | {index_to_label[index_inception]:30}")
Bernese_mountain_dog | Bernese_mountain_dog EntleBucher | EntleBucher Greater_Swiss_Mountain_dog | Appenzeller Appenzeller | Greater_Swiss_Mountain_dog basset | Gordon_setter
Jak widaÄ, klasy nieco siÄ różniÄ , ale TOP 1 pozostaje takie samo.
image, mask = explanation_alexnet.get_image_and_mask(
label=explanation_alexnet.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x242905e2030>
WyjaÅnienie dla AlexNet jak siÄ można byÅo spodziewaÄ - też siÄ różni, jednak w dalszym ciÄ
gu do klasyfikacji psa istotny jest... pies :)
Zadanie: porównaj predykcje obrazka dla modeli inception_v3 oraz alexnet¶
print("inception_v3")
get_prediction_probabilities(image_to_classify, inception_v3)
print()
print("alexnet")
get_prediction_probabilities(image_to_classify, alexnet)
inception_v3 Class: Bernese_mountain_dog | Probability: 0.935930 Class: EntleBucher | Probability: 0.038448 Class: Appenzeller | Probability: 0.023756 Class: Greater_Swiss_Mountain_dog | Probability: 0.001818 Class: Gordon_setter | Probability: 0.000009 Class: Blenheim_spaniel | Probability: 0.000007 Class: English_springer | Probability: 0.000002 Class: tabby | Probability: 0.000002 Class: robin | Probability: 0.000001 Class: guinea_pig | Probability: 0.000001 Class: amphibian | Probability: 0.000001 Class: Japanese_spaniel | Probability: 0.000001 Class: African_grey | Probability: 0.000001 Class: Brittany_spaniel | Probability: 0.000001 Class: toucan | Probability: 0.000001 alexnet Class: Bernese_mountain_dog | Probability: 0.400542 Class: EntleBucher | Probability: 0.208726 Class: Greater_Swiss_Mountain_dog | Probability: 0.194558 Class: Appenzeller | Probability: 0.186891 Class: basset | Probability: 0.002400 Class: Cardigan | Probability: 0.002184 Class: Border_collie | Probability: 0.001509 Class: kelpie | Probability: 0.000398 Class: bluetick | Probability: 0.000314 Class: Blenheim_spaniel | Probability: 0.000276 Class: collie | Probability: 0.000255 Class: borzoi | Probability: 0.000188 Class: Walker_hound | Probability: 0.000177 Class: Tibetan_mastiff | Probability: 0.000171 Class: English_springer | Probability: 0.000156
Zadanie domowe - wstÄp¶
W folderze data znajduje siÄĀ zdjÄcie amfibii:

amphibious_vehicle = get_image("./data/amfibia.jpg")
inception_v3 = models.inception_v3()
explanation_amhibious_vehicle_inception_v3 = explainer.explain_instance(
image=lime_transformer(amphibious_vehicle),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
image, mask = explanation_amhibious_vehicle_inception_v3.get_image_and_mask(
label=explanation_amhibious_vehicle_inception_v3.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=Inception_V3_Weights.IMAGENET1K_V1`. You can also use `weights=Inception_V3_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg)
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x2428a299640>
Model inception_v3 jak i jego wyjaÅnienie rzeczywiÅcie sugerujÄ
amfibiÄ jako najbardziej prawdopodobnÄ
klasÄ:
for index in explanation_amhibious_vehicle_inception_v3.top_labels:
print(index_to_label[index])
amphibian convertible racer car_wheel golfcart
Zadanie #1¶
Użyj dwóch różnych sieci neuronowych (poza inception_v3, którego przykÅad jest powyżej) do wygenerowania wyjaÅnieÅ.
(skorzystaj z moduÅu torchvision: https://pytorch.org/vision/stable/models.html)
alexNet¶
alexnet = models.alexnet()
explanation_amhibious_vehicle_alexnet = explainer.explain_instance(
image=lime_transformer(amphibious_vehicle),
classifier_fn=partial(predict_batch, alexnet),
top_labels=5,
num_samples=1000)
image, mask = explanation_amhibious_vehicle_alexnet.get_image_and_mask(
label=explanation_amhibious_vehicle_alexnet.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x24296462ff0>
for index in explanation_amhibious_vehicle_alexnet.top_labels:
print(index_to_label[index])
ashcan great_grey_owl pickup cardigan weevil
GoogleNet¶
googlenet = models.googlenet(pretrained=True)
explanation_amhibious_vehicle_googlenet = explainer.explain_instance(
image=lime_transformer(amphibious_vehicle),
classifier_fn=partial(predict_batch, googlenet),
top_labels=5,
num_samples=1000)
image, mask = explanation_amhibious_vehicle_googlenet.get_image_and_mask(
label=explanation_amhibious_vehicle_googlenet.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=GoogLeNet_Weights.IMAGENET1K_V1`. You can also use `weights=GoogLeNet_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg) Downloading: "https://download.pytorch.org/models/googlenet-1378be20.pth" to C:\Users\Monika/.cache\torch\hub\checkpoints\googlenet-1378be20.pth 100%|āāāāāāāāāā| 49.7M/49.7M [00:01<00:00, 50.1MB/s]
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x2428a29bef0>
for index in explanation_amhibious_vehicle_googlenet.top_labels:
print(index_to_label[index])
pickup tow_truck jeep amphibian convertible
Zadanie #2¶
Zmodyfikuj oryginalny obrazek w taki sposób, żeby najbardziej prawdopodobnÄ
klasÄ
dla każdej z tych sieci nie byÅa amfibia a jakiÅ inny pojazd (np. samochód). W tym celu możesz "zasÅoniÄ" czarnym kwadratem (wartoÅÄ 0 w macierzy reprezentujÄ
cej obraz) obszary istotne przy klasyfikacji.
PrzydatnÄ
rzeczÄ
bÄdzie skorzystanie z opcji hide_rest w funkcji get_image_and_mask i późniejsza obróbka obrazu
# Only the inception_v3 has the most probably amphibia
inception_v3 = models.inception_v3()
explanation_amhibious_vehicle_inception_v3 = explainer.explain_instance(
image=lime_transformer(amphibious_vehicle),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\inception.py:43: FutureWarning: The default weight initialization of inception_v3 will be changed in future releases of torchvision. If you wish to keep the old behavior (which leads to long initialization times due to scipy/scipy#11299), please set init_weights=True. warnings.warn(
0%| | 0/1000 [00:00<?, ?it/s]
image, mask = explanation_amhibious_vehicle_inception_v3.get_image_and_mask(
label=explanation_amhibious_vehicle_inception_v3.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x24298989e20>
modified_image = image.copy()
positive_mask = mask.astype(bool)
modified_image[positive_mask] = [0, 0, 0]
plt.imshow(modified_image)
plt.axis('off')
plt.show()
modified_image = Image.fromarray(modified_image)
explanation_amhibious_vehicle_inception_v3_modified = explainer.explain_instance(
image=lime_transformer(modified_image),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
for index in explanation_amhibious_vehicle_inception_v3_modified.top_labels:
print(index_to_label[index])
cairn bulbul coral_fungus tench toilet_tissue
Zadanie #3¶
Ponownie zmodyfikuj oryginalny obraz, ale tym razem zaszumiajÄ c go w losowy sposób (przykÅadowa implementacja: https://www.geeksforgeeks.org/add-a-salt-and-pepper-noise-to-an-image-with-python/). Czy najbardziej prawdopodobna klasa zmienia siÄ wraz ze zmianÄ szumu? Przetestuj dla każdego z modeli.
import random
import cv2
def add_noise(img):
row , col = img.shape
# white
number_of_pixels = random.randint(300, 10000)
for i in range(number_of_pixels):
y_coord=random.randint(0, row - 1)
x_coord=random.randint(0, col - 1)
img[y_coord][x_coord] = 255
# black
number_of_pixels = random.randint(300 , 10000)
for i in range(number_of_pixels):
y_coord=random.randint(0, row - 1)
x_coord=random.randint(0, col - 1)
img[y_coord][x_coord] = 0
return img
img = cv2.imread('./data/amfibia.jpg',
cv2.IMREAD_GRAYSCALE)
cv2.imwrite('salt-and-pepper-amfibia.jpg',
add_noise(img))
True
amphibious_noise = get_image('salt-and-pepper-amfibia.jpg')
plt.imshow(amphibious_noise)
plt.axis('off')
plt.show()
Inception_v3¶
inception_v3 = models.inception_v3()
explanation_amhibious_vehicle_inception_v3 = explainer.explain_instance(
image=lime_transformer(amphibious_noise),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
image, mask = explanation_amhibious_vehicle_inception_v3.get_image_and_mask(
label=explanation_amhibious_vehicle_inception_v3.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
plt.axis('off')
plt.show()
for index in explanation_amhibious_vehicle_inception_v3.top_labels:
print(index_to_label[index])
Cardigan bulbul coral_fungus tench toilet_tissue
AlexNet¶
alexnet = models.alexnet()
explanation_amhibious_vehicle_alexnet = explainer.explain_instance(
image=lime_transformer(amphibious_noise),
classifier_fn=partial(predict_batch, alexnet),
top_labels=5,
num_samples=1000)
image, mask = explanation_amhibious_vehicle_alexnet.get_image_and_mask(
label=explanation_amhibious_vehicle_alexnet.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
plt.axis('off')
plt.show()
0%| | 0/1000 [00:00<?, ?it/s]
for index in explanation_amhibious_vehicle_alexnet.top_labels:
print(index_to_label[index])
brambling cricket English_setter triumphal_arch head_cabbage
GoogLeNet¶
googlenet = models.googlenet(pretrained=True)
explanation_amhibious_vehicle_googlenet = explainer.explain_instance(
image=lime_transformer(amphibious_noise),
classifier_fn=partial(predict_batch, googlenet),
top_labels=5,
num_samples=1000)
image, mask = explanation_amhibious_vehicle_googlenet.get_image_and_mask(
label=explanation_amhibious_vehicle_googlenet.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( c:\Users\Monika\AppData\Local\Programs\Python\Python312\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=GoogLeNet_Weights.IMAGENET1K_V1`. You can also use `weights=GoogLeNet_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg)
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x2429e77ee40>
for index in explanation_amhibious_vehicle_googlenet.top_labels:
print(index_to_label[index])
amphibian jeep tow_truck snowplow half_track
Using various models the most probably class is changing. Only GoogLeNet still is finding amphibian as first choice and futher other car types.